The statistical mechanics of turbo codes

The "turbo codes", recently proposed by Berrou et. al. are written as a disordered spin Hamiltonian. It is shown that there is a threshold Theta such that for signal to noise ratios v^2 / w^2 > Theta, the error probability per bit vanishes in the thermodynamic limit, i.e. the limit of infinitly long sequences. The value of the threshold has been computed for two particular turbo codes. It is found that it depends on the code. These results are compared with numerical simulations.Comment: 23 pages, 6 figures: Fig.2 has been replaced (in the preceding version it was identical to Fig.1

Scale Invariance and Self-averaging in disordered systems

In a previous paper we found that in the random field Ising model at zero temperature in three dimensions the correlation length is not self-averaging near the critical point and that the violation of self-averaging is maximal. This is due to the formation of bound states in the underlying field theory. We present a similar study for the case of disordered Potts and Ising ferromagnets in two dimensions near the critical temperature. In the random Potts model the correlation length is not self-averaging near the critical temperature but the violation of self-averaging is weaker than in the random field case. In the random Ising model we find still weaker violations of self-averaging and we cannot rule out the possibility of the restoration of self-averaging in the infinite volume limit.Comment: 7 pages, 4 ps figure

Scale Invariance in disordered systems: the example of the Random Field Ising Model

We show by numerical simulations that the correlation function of the random field Ising model (RFIM) in the critical region in three dimensions has very strong fluctuations and that in a finite volume the correlation length is not self-averaging. This is due to the formation of a bound state in the underlying field theory. We argue that this non perturbative phenomenon is not particular to the RFIM in 3-d. It is generic for disordered systems in two dimensions and may also happen in other three dimensional disordered systems

Thouless-Anderson-Palmer Approach for Lossy Compression

We study an ill-posed linear inverse problem, where a binary sequence will be reproduced using a sparce matrix. According to the previous study, this model can theoretically provide an optimal compression scheme for an arbitrary distortion level, though the encoding procedure remains an NP-complete problem. In this paper, we focus on the consistency condition for a dynamics model of Markov-type to derive an iterative algorithm, following the steps of Thouless-Anderson-Palmer's. Numerical results show that the algorithm can empirically saturate the theoretical limit for the sparse construction of our codes, which also is very close to the rate-distortion function.Comment: 10 pages, 3 figure

Transient dynamics for sequence processing neural networks

An exact solution of the transient dynamics for a sequential associative memory model is discussed through both the path-integral method and the statistical neurodynamics. Although the path-integral method has the ability to give an exact solution of the transient dynamics, only stationary properties have been discussed for the sequential associative memory. We have succeeded in deriving an exact macroscopic description of the transient dynamics by analyzing the correlation of crosstalk noise. Surprisingly, the order parameter equations of this exact solution are completely equivalent to those of the statistical neurodynamics, which is an approximation theory that assumes crosstalk noise to obey the Gaussian distribution. In order to examine our theoretical findings, we numerically obtain cumulants of the crosstalk noise. We verify that the third- and fourth-order cumulants are equal to zero, and that the crosstalk noise is normally distributed even in the non-retrieval case. We show that the results obtained by our theory agree with those obtained by computer simulations. We have also found that the macroscopic unstable state completely coincides with the separatrix.Comment: 21 pages, 4 figure

Typical performance of low-density parity-check codes over general symmetric channels

Typical performance of low-density parity-check (LDPC) codes over a general binary-input output-symmetric memoryless channel is investigated using methods of statistical mechanics. Theoretical framework for dealing with general symmetric channels is provided, based on which Gallager and MacKay-Neal codes are studied as examples of LDPC codes. It has been shown that the basic properties of these codes known for particular channels, including the property to potentially saturate Shannon's limit, hold for general symmetric channels. The binary-input additive-white-Gaussian-noise channel and the binary-input Laplace channel are considered as specific channel noise models.Comment: 10 pages, 4 figures, RevTeX4; an error in reference correcte

Phase Transitions in Disordered Systems: The Example of the Random-Field Ising Model in Four Dimensions

By performing a high-statistics simulation of the D=4 random-field Ising model at zero temperature for different shapes of the random-field distribution, we show that the model is ruled by a single universality class. We compute to a high accuracy the complete set of critical exponents for this class, including the correction-to-scaling exponent. Our results indicate that in four dimensions (i) dimensional reduction as predicted by the perturbative renormalization group does not hold and (ii) three independent critical exponents are needed to describe the transition

Typical Performance of Gallager-type Error-Correcting Codes

The performance of Gallager's error-correcting code is investigated via methods of statistical physics. In this approach, the transmitted codeword comprises products of the original message bits selected by two randomly-constructed sparse matrices; the number of non-zero row/column elements in these matrices constitutes a family of codes. We show that Shannon's channel capacity is saturated for many of the codes while slightly lower performance is obtained for others which may be of higher practical relevance. Decoding aspects are considered by employing the TAP approach which is identical to the commonly used belief-propagation-based decoding.Comment: 6 pages, latex, 1 figur

Community Detection as an Inference Problem

We express community detection as an inference problem of determining the most likely arrangement of communities. We then apply belief propagation and mean-field theory to this problem, and show that this leads to fast, accurate algorithms for community detection.Comment: 4 pages, 2 figure

Cryptographical Properties of Ising Spin Systems

The relation between Ising spin systems and public-key cryptography is investigated using methods of statistical physics. The insight gained from the analysis is used for devising a matrix-based cryptosystem whereby the ciphertext comprises products of the original message bits; these are selected by employing two predetermined randomly-constructed sparse matrices. The ciphertext is decrypted using methods of belief-propagation. The analyzed properties of the suggested cryptosystem show robustness against various attacks and competitive performance to modern cyptographical methods.Comment: 4 pages, 2 figure